Image visual saliency analysis algorithms have achieved satisfactory performance on existing datasets, but these datasets have two major problems. Firstly, most of the images contain only one salient object. Secondly, users' cognition of multiple salient objects in the same image was ignored during building salient objects' ground truth. The above problems result in that the performance of saliency analysis algorithms used in the real applications cannot be reflected by the evaluation on the existing datasets. So in this paper, a novel method of labeling the ground truth of salient objects was proposed. Firstly, a software to collect users' cognition of the important values of multiple salient objects in each image was designed and implemented. Then, according to the collected data from each user, the ground truth map represented as a gray scale image was created by manually labeling the regions covered by the salient objects. The pixel value of each region equals to the collected saliency in the first step. Based on the improved ground truth labeling method, a salient object dataset contains 1000 multi-object images was built. A ground truth map for each image was created to record users' cognition of the objects' saliencies. Then 10 state-of-the-art saliency analysis algorithms on existing datasets and the established dataset were compared. The experimental results show that these algorithms' performances are greatly reduced on the established dataset, such as the Area Under Curve of Receiver-Operating Characteristic (ROC-AUC) has a greatest decline of more than 0.5. The results prove the problems of existing datasets and the demand of building a new dataset, and point out the insufficiency of saliency analysis algorithms on complex images with multiple salient objects.